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[Type of the Document] Specification 

[Title of the Invention] IMAGE PROCESSING APPARATUS 

AND 

METHOD 

5 [What Is Claimed Is:] 

[Claim 1] An image processing apparatus, 
characterized by comprising 

first input means for inputting an image signal, 
second input means for inputting position 
10 information indicating an arbitrary position of an 
image and image data in the arbitrary position, 

extracting means for extracting the image data in 
the position corresponding to the position information 
from the image signal input by said first input means, 
15 setting means for setting an image processing 

parameter on the basis of the image data extracted by 
said extracting means and the image data input by said 
second input means, and 

processing means for processing the image signal 
20 input by said first input means by using the image 
processing parameter set by said setting means . 

[Claim 2] The image processing apparatus 
according to claim 1, characterized in that said first 
input means inputs an image signal output from image 
25 sensing means. 

[Claim 3] The image processing apparatus 
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according to claim 1, characterized in that said 
setting means sets an image processing parameter for 
converting the image data extracted by said extracting 
means into the image data input by said second input 
5 means . 

[Claim 4] The image processing apparatus 
according to claim 1, characterized in that said 
processing means performs color balance processing by 
using the image processing parameter set by said 
10 setting means. 

[Claim 5] The image processing apparatus 
according to claim 1, characterized in that said 
processing means perfoirms white balance processing when 
said setting means sets no image processing parameter. 
15 [Claim 6] An image processing method, 

characterized by comprising 

the first input step of inputting an image signal, 
the second input step of inputting position 
information indicating an arbitrary position of an 
20 image and image data in the arbitrary position, 

the extraction step of extracting the image data 
in the position corresponding to the position 
information from the image signal in the first input 
step, 

25 the setting step of setting an image processing 

parameter on the basis of the image data extracted in 
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the extraction step and the image data input in the 
second input step, and 

the processing step of processing the image signal 
input in the first input step by using the image 
5 processing parameter set in the setting step, 

[Claim 7] An image processing appeiratus 
characterized in that said apparatus adjusts a hue of 
an input image signal, and comprises 

input means for inputting color information with 
10 respect to a specific region which forms a part of an 
image, and 

determining means for determining a hue of a whole 
image represented by the image signal on the basis of 
the color information. 
15 [Detailed Description of the Invention] 
[00011 

[Technical Field of the Invention] 
The present invention relates to an image 
processing apparatus and method and, for example, an 
20 image processing apparatus and method of adjusting the 
hue of an input image signal, 
10002] 

[Prior Art] 

In a television camera using an image sensing 
25 device such as an image sensing tube or a CCD, the 
level of a color signal consisting of a plurality of 
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components constituting an image signal output from the 
Image sensing device must be adjusted such that the 
respective components are balanced with each other. 
[0003] 

5 In order to achieve the above object, so-called 

white balance adjustment is normally performed in this 
type of television camera so that an object which is 
supposed to look white looks white. 
[0004] 

10 Generally, the white balance adjustment amplifies 

■ and corrects a plurality of color component signals 
constituting an image signal. Consequently, the output 
levels of the color component signals are so adjusted 
as to be equal to each other with respect to an object 

15 which is supposed to look white. 
[0005] 

[Problems That the Invention Is to Solve] 
However, the above- described tectmique has the 
following problems. The method of adjusting the hue of 

20 an image by adjusting the white balance is effective 
when a sufficient number of objects which are supposed 
to look white exist in an image signal obtained from an 
image sensing device or the like. However, no such 
object which is supposed to look white exists in an 

25 image signal or only a very few such objects exist in 
an image signal in many instances. In these instances. 
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therefore, it is in principle impossible to adjust the 

hue by adjusting the white balance. 

[0006] 

The present invention has been made to solve the 
5 above problems, and has as its object to provide an 
image processing apparatus and method capable of 
adjusting a color balance even when no object which is 
supposed to look white exists or only a few such 
objects exist - 
10 [0007] 

[Means of Solving the Problems] 
The present invention has the following 
configuration as one means to achieve the above object • 
[0008] 

15 An image processing apparatus according to the 

present invention is characterized by comprising first 
input means for inputting an image signal, second input 
means for inputting position information indicating an 
arbitrary position of an image and image data in the 

20 arbitrary position, extracting means for extracting the 
image data in the position corresponding to the 
position information from the image signal input by the 
first input means, setting means for setting an image 
processing parameter on the basis of the image data 

25 extracted by the extracting means and the image data 
input by the second input means, and processing means 
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for processing the Image signal input by the first 
input means by using the image processing parameter set 
by the setting means. 
I 0009] 

5 Also, an image processing apparatus for adjusting 

a hue of an input image signal is characterized by 
comprising input means for inputting color information 
with respect to a specific region which foonns a part of 
an image, and determining means for determining a hue 
10 of a whole image represented by the image signal on the 
basis of the color information, 
[0010] 

An image processing method according to the 
present invention is characterized by comprising the 

15 first input step of inputting an image signal, the 
second input step of inputting position information 
indicating an arbitrary position of an image and image 
data in the arbitrary position, the extraction step of 
extracting the image data in the position corresponding 

20 to the position information from the image signal input 
in the first input step, the setting step of setting an 
image processing parameter on the basis of the image 
data extracted in the extraction step and the image 
data input in the second input step, and the processing 

25 step of processing the image signal input in the first 
input step by using the image processing parameter set 
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in the setting step* 
[0011] 

[Embodiments] 
An image processing apparatus according to an 
5 embodiment of the present invention will be described 
below in detail with reference to the accompanying 
drawings . 
[0012] 

Fig, 1 is a block diagram showing the constitution 
10 of an image processing apparatus according to an 
embodiment of the present invention. 
[0013] 

Referring to Fig* 1, reference numeral 10 denotes 
a color image sensing unit, e.g., a digital camera 

15 including an image sensing device such as an image 

sensing tube or a CCD. The color image sensing unit 10 
senses the image of an object and outputs two- 
dimensional digital data as the image sensing data of 
the object. Reference niomeral 20 denotes a color image 

20 reproduction processor for performing image 

reproduction processing for the output two-dimensional 
digital data transmitted from the color image sensing 
unit 10. For example, the color image reproduction 
processor 20 converts the two-dimensional digital data 

25 into digital NTSC-RGB data and outputs the converted 
data. Reference numeral 30 denotes a color image 
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reproducing display consisting of, e.g., a color video 
card and a monitor. The color image reproducing 
display 30 receives the output color image signal from 
the color image reproduction processor 20 and displays 
5 it as a color image on the monitor, 
[ 0014] 

Although not shown in Fig. 1, the apparatus of 
this embodiment can be connected to a printer which 
receives the output color image signal from the color 
10 image reproduction processor 20 and records the color 
image on a recording medium. The apparatus can also be 
connected to an image storage device or a personal 
computer as well as to a printer. 
[0015] 

15 The color image reproduction processor 20 performs 

various processes necessary for color image 
reproduction. In the following description, processing 
of color balance adjustment performed by the color 
image reproduction processor 20 and the configuration 

20 for the processing will be explained. Normal color 
image reproduction requires various processes and 
components not clearly explained and illustrated in the 
drawings. However, these processes and components are 
well known to those skilled in the art, so a detailed 

25 description thereof will be omitted. 
[0016] 
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The color image reproduction processor 20 includes 
the following components. That is, reference numeral 
21 denotes an image sensing data holding mechanism for 
holding the output digital image sensing data from the 
5 color image sensing unit 10; 22, an image sensing data 
extracting mechanism for extracting a portion of the 
image sensing data; and 24, an image position 
indicating mechanism for indicating an image position 
of the imag^ sensing data to be extracted by the image 
10 sensing data extracting mechanism 22. 
[0017] 

Reference numeral 25 denotes an image data input 
mechanism for inputting data that the image data in the 
indicated position finally takes. Reference numeral 23 

15 denotes an image reproduction parameter determining 
mechcinism for determining an image reproduction 
parameter by using the input image data from the image 
data input mechanism 25 and the image sensing data 
extracted by the image sensing data extracting 

20 mechanism 22. A color balance adjusting mechanism 26. 
adjusts the color balance of the image sensing data, 
when an image is reproduced, by using the image 
reproduction parameter determined by the image 
reproduction peirameter determining mechanism 23. 

25 [0018] 

Reference nximeral 27 denotes a control unit 
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consisting of, e.g., a CPU, a program ROM, a work RAM, 
and an I/O device. The control unit 27 executes the 
color image reproduction processing by controlling the 
above components and components not shown in Fig. 1. 
[0019] 

The operation of this embodiment will be described 
below. Fig. 2 is a flow chart showing the operation of 
the color image reproduction processor 20, and 
particularly the procedure of color balance adjustment. 
This procedure is started by the control unit 27 when 
the input digital image sensing data from the color 
image sensing unit 10 is held in the image sensing data 
holding mechanism 21. 
10020] 

In step SI, the control unit 27 checks whether an 
image position is indicated by the image position 
indicating mechanism 24. If no image position is 
indicated, the flow advances to step Sll, and the color 
balance adjusting mechanism 26 performs normal white 
balance processing for the image sensing data held in 
the image sensing data holding mechanism 21, thereby 
obtaining reproduced image data. Thereafter, the 
processing is completed. On the other hand, if an 
image position is indicated, the indicated image 
position (to be referred to as an "indicated position" 
hereinafter) is sent to the image sensing data 
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extracting mechanism 22, and the flow advances to step 
S2 . In step S2, it is checked whether the image data 
input mechanism 25 inputs image data. 
[0021] 

An example of the white balance processing usually 
performjed will be described below. Note that this 
white balance processing is done on the basis of image 
sensing data. To perform the white balance processing, 
it is necessary to extract light source information 
from image sensing data. Generally, the color balance 
is adjusted on the basis of the extracted light source 
information, and the white balance processing is 
performed such that the color of the light source 
becomes white. Accordingly, if the light source 
information is not correctly extracted, incorrect white 
balance processing is performed. 
[0022] 

An example of the method of extracting the light 
source information from image sensing data is as 
follows. It is generally considered that in an image 
sensed by using reflected light from an object, a 
region (white region) having the light source 
information has a high luminance. Therefore, each 
color data in a region having the highest luminance in 
image sensing data is used as the light source 
information. This method is effective when all regions 
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in image sensing data are obtained by using reflected 
light from an object and a white region is contained in 
the image. However, if an image has no white region or 
a luminescent body is included in the image, a region 
having a high luminance may not necessarily have the 
light source information. If this is the case, it is 
sometimes not possible to accurately extract the light 
source information. 
[0023] 

A method other than the above method is to obtain 
the light source information by calculating the average 
of image sensing data, i.e., all pixels, constituting 
an image, for each color. This method is useful when 
all colors in an image are evenly present with no 
difference between them. Generally, however, a large 
amount of a certain specific color component often 
exists, and in this case it is not possible to 
accurately obtain the light source information. 
[0024] 

It is ]cnown that common light sources excluding 
some special light sources exist in a certain 
chromaticity range. If the characteristics of a color 
filter used in photography are known, the range of the 
existence of image sensing data having the light source 
information can be set from the above relationship. 
Accordingly, image sensing data can be extracted more 
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accurately by sampling image sensing data in this range 
of existence from image sensing data constituting an 
image. 
[0025] 

More specifically, assxime that color data obtained 
as image sensing data are {Ma,Gr, Ye,Cy) . These data 
are normalized by the following equations: 

Kmg = (Ma - Gr)/Gr 

Kyc = (Ye - Cy)/Cy 
[00261 

With respect to the light source, Kmg and Kyc fall 
within a certain range in accordance with the above 
relations. Therefore, data (Ma,Gr , Ye^Cy) in this range 
are stored as data of the light source information and 
averaged (i.e., the total sum of each color data of 
image sensing data meeting the condition is divided by 
the total nximber of sampling cycles). In this way, the 
light source information can be accurately obtained. 
[0027] 

The above method is performed singly, or in 
combination with another, as the white balance 
processing, and particularly as the method of 
extracting the light source information in the white 
balance processing. 
[0028] 

If no image data is input in step S2, the flow 
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advances to step S12. In step S12, the image sensing 
data extracting mechanism 22 extracts image sensing 
data corresponding to the indicated position and 
supplies the extracted image sensing data (to be 
5 referred to as "extracted data" hereinafter) to the 
image reproduction parameter determining mechanism 23. 
Upon receiving the extracted data, the image 
reproduction peirameter determining mechanism 23 
determines an image reproduction parameter by using the 

10 extracted data such that the reproduced image data in 
the indicated position becomes image data indicating 
white. The image reproduction parameter determining 
mechanism 23 supplies the deterimined image reproduction 
parameter to the color balance adjusting mechanism 26. 

15 [0029] 

On the other hand, if image data is input by the 
image data input mechanism 25, this image data (to be 
referred to as "designated data" hereinafter) is 
supplied to the image reproduction parameter 

20 determining mechanism 23, and the flow advances to step 
S3. In step S3, the image sensing data extracting 
mechanism 22 extracts image sensing data corresponding 
to the Indicated position and supplies the extracted 
data to the image reproduction parameter determining 

25 mechanism 23. Upon receiving the designated data and 
the extracted data, the image reproduction parameter 
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determining mechanism 23 determines an image 
reproduction parameter by using the designated data and 
the extracted data such that the reproduced image data 
in the indicated position becomes the designated data. 
5 The image reproduction parameter determining mechanism 
23 supplies the determined image reproduction parameter 
to the color balance adjusting mechanism 26. 
[0030] 

In step S4, the color balance adjusting mechanism 
10 26 adjusts the color balance of the image sensing data 
held in the image sensing data holding mechanism 21 by 
using the image reproduction parameter obtained in step 
S3 or S12, thereby obtaining reproduced image data. 
Thereafter, the control unit 27 completes the 
15 processing • 
[0031] 

The reproduced image data thus obtained is 
supplied to the color image reproducing display 30 
which in turn reproduces and displays the color image. 
20 [0032] 

In the above procedure, the coordinate positions 
of image sensing data on a two-dimensional plane and 
the coordinate positions of a reproduced image obtained 
by performing image reproduction processing for the 
25 image sensing data are indicated, and the reproduced 
image data (indicated data) in this indicated position 
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is supplied. Consequently, an image reproduction 
parameter is automatically determined on the basis of 
the image sensing data and the indicated data in this 
indicated position. Image reproduction processing is 
5 performed by using the image reproduction parameter 

thus determined. Accordingly, a reproduced image whose 
color balance is properly adjusted can be obtained with 
a simple operation. 
[0033] 

10 In particular, this embodiment is not limited to 

the color balance adjustment (white balance adjustment) 
based on white; that is, the color balance can be 
adjusted by using any arbitrary color. Therefore, 
objects applicable to the color balance adjustment are 

15 not limited, i.e., any arbitrary object can be selected. 
[0034] 

One example of the method of automatically 
determining an image reproduction parameter will be 
described below. 
20 [0035] 

Assume that image sensing data in a certain 
indicated position is obtained as (Mg,Gr, Ye,Cy) from 
the output of an image sensing device and image 
reproduction data (R,G,B) is obtained by performing the 
25 following conversion for the image sensing data 
(Ma,Gr,Ye,Cy) . 
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•-•(1) 
[0036] 

In the above equation, kl, k2, k3, and k4 are 
5 parameters for color balance adjustment, i.e., color 

balance adjustment gains in the data Mg, Gr, Cy, and Ye. 
Commonly, the default values of all these parameters 
are 1, emd all, al2,^.., a34 are conversion matrix 
coefficients for converting {Mg,Gr,Cy, Ye) into (R,G,B), 

10 These conversion matrix coefficients are normally 
determined on the basis of the spectral sensitivity 
characteristics of an image sensing device used in 
photography or the characteristics of a colorimetric 
system used as an output image signal, and are 

15 generally fixed by setting. 
[0037] 

Commonly, the color balance adjustment is done by 
using the parameters kl, k2, k3, and k4. For example, 
assuming that image reproduction data in a certain 
20 image position is (R',G',B')/ the parameters kl, k2, k3, 
and k4 are determined by the following equation such 
that image sensing data (Mg* ,Gr ' ,Cy ' , Ye ' ) is converted 
into image reproduction data (R',G',B'). In the 
following equation, the parameter kl is set beforehand 
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as a constant, and an important factor In the color 



By the use of the above method, the image 
reproduction parameters kl, k2, k3, and k4 can be 
determined by using the image sensing data 
(Mg' ,Gr ■ ,Cy' , Ye' ) in a certain indicated position and 
the designated data {R',G',B'). 
[0039] 

[Modifications ] 

In the above embodiment, the image position 
indicating mechanism 24 can be so designed that an 
operator inputs numerical values (coordinates) by using 
a keyboard. Alternatively, dummy reproduced image data 
is obtained by performing image reproduction processing 
(e.g,, white balance processing) using a default image 
reproduction parameter for the image sensing data held 
in the image sensing data holding mechanism 21, and 
displayed on, e.g,, a monitor. In this case, an 
indicated position can be input by using a pointing 
device such as a light pen, a mouse, or a touch panel 
and a pointing mechanism. This further facilitates the 
operation. 



balance adjustment is the parameter ratio. 




...(2) 



[0038] 
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[0040] 

Furthermore, the value of color information (image 
data in an indicated position) in a particular region 
can be directly input from, e.g., a keyboard. 
Alternatively, it is possible to display a color 
palette previously stored in a ROM or the like on a 
monitor and allows an operator to choose a desired 
color from the palette. This further facilitates the 
operation. Also, a so-called color picker can be 
displayed on a monitor if an internal calculating unit 
is used. In this case, by using expressions 
representing the attributes of a color such as the hue, 
saturation, and lightness, it is possible to supply 
color information by using all or some of these 
attributes, 
[0041] 

The color information can also be conceptually 
supplied by using words or symbols. This is effective 
when there is a color that the user wants to reproduce 
more beautifully or to emphasize, i.e., when image 
reproduction is performed by using a stored color such 
as "color of sky", "color of leaf", "color of sea", or 
"color of skin". 
10042] 

As described above, in this embodiment, the 
position of an image is designated, color information 
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pertaining to the designated position is input, and an 
image reproduction parameter is determined on the basis 
of image sensing data corresponding to the designated 
position and the input color information. Reproduced 
5 image data is obtained by performing image reproduction 
processing for the image sensing data by using the 
determined image reproduction parameter. Accordingly, 
the color balance of the whole image can be adjusted by 
using an object of an arbitrary color in an arbitrary 

10 position on the screen, for example. This solves the 

problem of white balance adjustment that the adjustment 
is impossible when an image has no white object or only 
a very few white objects exist in an image. Also, by 
supplying a color (e.g., "color of skin") that the user 

15 wants to reproduce more accurately, as color 

information, it is possible to accurately reproduce a 
place where the user wants to reproduce and a color 
belonging to the place, or a color that the user wants 
to reproduce. 

20 [0043] 

[Other Embodiments] 

The object of the present invention is also 
achieved by supplying a recording medium on which the 
program code of software for realizing the function of 

25 the above -described embodiment is recorded to a system 
or an apparatus and reading out and executing the 
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program code stored in the recording medium by using a 
computer (or a CPU or MPU) of the isystem or apparatus, 
of course. In this case, the program code itself read 
out from the recording medium realizes the novel 
function of the present invention and the recording 
medium adapted to provide the program code constitutes 
the present invention. Examples of a recording medium 
for providing the program code include a floppy disk, a 
hard disk, an optical disk, a magnetooptical disk, a 
CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory 
card, a ROM, and the like. 
[0044] 

Note that the present invention can be practiced 
in various forms as shown in the above -described 
plurality of embodiments. Therefore, the present 
invention can be theorized or implemented in software 
utilizing various information without departing from 
the scope and spirit of the present invention as 
described above. Alternatively, the present invention 
can be realized in algorithm without departing from the 
scope and spirit of the present invention as described 
above and can be applied to hardware or an apparatus 
which operates in accordance with the algorithm. 
[0045] 

[Effect of the Invention] 
As described above, according to the present 
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invention, an image processing apparatus and method 
capable of adjusting a color balance even when no 
object which is supposed to look white exists or only a 
few such objects exist can be provided. 
5 [Brief Description of the Drawings] 
[Fig. 1] 

Fig. 1 is a block diagram showing the configuration of 
an image processing apparatus according to an 
embodiment of the present invention. 
10 [Fig. 2] 

Fig. 2 is a flow chart showing an example of the 
operation of the color image reproduction processing 
unit shown in Fig. 1. 

[Description of the Reference Numerals] 



15 10 color image sensing unit 

20 color image reproduction processor 

21 image sensing data holding mechanism 

22 image sensing data extracting mechanism 
24 image position indicating mechanism 

20 25 image data input mechanism 

23 image reproduction parameter determining 
mechanism 

26 color balance adjusting mechanism 

27 control unit 

25 30 color image reproducing display 
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[Type of the Document] Abstract 
[Abstract] 

[Problem] It Is an object of the present Invention to 
provide an image processing apparatus and method 
capable of adjusting a color balance even when no 
object which is supposed to look white exists or only a 
few such objects exist. 

[Solving Means] An image position indicating mechanism 
24 indicates an cirbitrary position of an image and an 
image data input mechanism 25 inputs image data in the 
indicated position. An image data extracting mechanism 
22 extracts image data corresponding to the indicated 
position from image sensing data held in an image 
sensing data holding mechanism 21. An image 
reproduction parameter determining mechanism 23 
determines an image reproduction parameter on the basis. 
of the extracted image data and the input image data. 
A color balance adjusting mechanism 26 performs color 
balance adjustment for the image sensing data held in 
the image sensing data holding mechanism 21 by using 
the determined image reproduction parameter. 
[Selected Drawing] Pig. 1 
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